Skip to content

Increase total fields limit from 2000 to 2500#2584

Merged
trisch-me merged 4 commits intomainfrom
feature/update-field-limit
Mar 16, 2026
Merged

Increase total fields limit from 2000 to 2500#2584
trisch-me merged 4 commits intomainfrom
feature/update-field-limit

Conversation

@Mpdreamz
Copy link
Member

@Mpdreamz Mpdreamz commented Feb 17, 2026

ECS 9.3.0 Specification Analysis

Total Field Count

Version Fields Multi-fields Total
v9.3.0 2,141 123 2,264
v9.0.0 1,848 91 1,939
Delta +293 +32 +325 (16.8%)

Special Field Types

Type v9.0.0 v9.3.0 Change
object (container/grouping) 15 90 +75 (+500%)
Fields with multi_fields 91 123 +32

New Field Sets in v9.3.0

Field Set Fields
entity 26
gen_ai 26
Total 52

Field Sets with Significant Growth

Field Set v9.0.0 v9.3.0 Growth % Change
user 46 116 +70 +152%
host 42 110 +68 +162%
cloud 33 72 +39 +118%
service 33 72 +39 +118%
orchestrator 15 28 +13 +87%
device 5 10 +5 +100%
process 389 391 +2 +0.5%
threat 435 437 +2 +0.5%

Largest Field Sets (v9.3.0)

Field Set Fields % of Total
threat 437 19.3%
process 391 17.3%
file 144 6.4%
user 116 5.1%
host 110 4.9%
tls 77 3.4%
cloud 72 3.2%
service 72 3.2%
dll 46 2.0%
observer 41 1.8%

Summary

  • 54 total field sets: 2 new, 6 with growth, 46 unchanged
  • Growth drivers: new field sets (entity, gen_ai) account for 52 fields; existing field set expansions (user, host, cloud, service, orchestrator, device) account for 241 fields; 32 new multi-field variants
  • The object type count jumped from 15 to 90 — these are the entity sub-objects added to user, host, cloud, service, and orchestrator for the new entity reuse pattern
  • The current composable template ships with total_fields.limit: 2000, which is insufficient for the 2,264 fields in v9.3.0

We should probably look into generating mapping templates instead, so users only map the fields they actually use.

# ECS 9.3.0 Specification Analysis

## Total Field Count

| Version | Fields | Multi-fields | Total |
|---------|--------|--------------|-------|
| v9.3.0 | 2,141 | 123 | **2,264** |
| v9.0.0 | 1,848 | 91 | **1,939** |
| **Delta** | **+293** | **+32** | **+325 (16.8%)** |

## Special Field Types

| Type | v9.0.0 | v9.3.0 | Change |
|------|--------|--------|--------|
| `object` (container/grouping) | 15 | 90 | +75 (+500%) |
| `nested` (array-type) | 25 | 28 | +3 |
| Fields with `multi_fields` | 91 | 123 | +32 |

## New Field Sets in v9.3.0

| Field Set | Fields |
|-----------|--------|
| `entity` | 26 |
| `gen_ai` | 26 |
| **Total** | **52** |

## Field Sets with Significant Growth

| Field Set | v9.0.0 | v9.3.0 | Growth | % Change |
|-----------|--------|--------|--------|----------|
| `user` | 46 | 116 | +70 | +152% |
| `host` | 42 | 110 | +68 | +162% |
| `cloud` | 33 | 72 | +39 | +118% |
| `service` | 33 | 72 | +39 | +118% |
| `orchestrator` | 15 | 28 | +13 | +87% |
| `device` | 5 | 10 | +5 | +100% |
| `process` | 389 | 391 | +2 | +0.5% |
| `threat` | 435 | 437 | +2 | +0.5% |

## Largest Field Sets (v9.3.0)

| Field Set | Fields | % of Total |
|-----------|--------|------------|
| `threat` | 437 | 19.3% |
| `process` | 391 | 17.3% |
| `file` | 144 | 6.4% |
| `user` | 116 | 5.1% |
| `host` | 110 | 4.9% |
| `tls` | 77 | 3.4% |
| `cloud` | 72 | 3.2% |
| `service` | 72 | 3.2% |
| `dll` | 46 | 2.0% |
| `observer` | 41 | 1.8% |

## Summary

- **54 total field sets**: 2 new, 6 with growth, 46 unchanged
- **Growth drivers**: new field sets (`entity`, `gen_ai`) account for 52 fields; existing field set expansions (`user`, `host`, `cloud`, `service`, `orchestrator`, `device`) account for 241 fields; 32 new multi-field variants
- The `object` type count jumped from 15 to 90 — these are the entity sub-objects added to `user`, `host`, `cloud`, `service`, and `orchestrator` for the new `entity` reuse pattern
- The current composable template ships with `total_fields.limit: 2000`, which is insufficient for the 2,264 fields in v9.3.0
@Mpdreamz Mpdreamz requested a review from a team as a code owner February 17, 2026 15:53
@github-actions
Copy link

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@andrewkroh
Copy link
Member

We should probably look into generating mapping templates instead, so users only map the fields they actually use.

Like having an ecs_process and ecs_threat component template available to pick and choose as needed? That the idea?

@trisch-me
Copy link
Contributor

That’s a nice catch, we should probably have this automated

@kgeller
Copy link
Contributor

kgeller commented Mar 10, 2026

Great catch @Mpdreamz ! I saw the build was failing due to not having generated some of the artifacts, so I took care of that and added a changelog entry. You should now be good to merge whenever you're ready

@Mpdreamz
Copy link
Member Author

Ty @kgeller!

@trisch-me trisch-me merged commit d45897f into main Mar 16, 2026
9 checks passed
@trisch-me
Copy link
Contributor

@kgeller I think this might be one of low hanging fruits to add for agent check

@kgeller kgeller deleted the feature/update-field-limit branch March 16, 2026 18:36
@kgeller
Copy link
Contributor

kgeller commented Mar 16, 2026

@trisch-me 100%. I think it fits very nicely into the "Scheduled repo audits" part of the section 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants